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DETAILED ACTION 

1 . This office action is responsive to application 1 0/702200 filed on November 4, 
2003. Claims 1-15 are pending in the application and have been examined by the 
examiner. 

Priority 

2. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 19(a)-(d), which 
papers have been placed of record in the file. 

Claim Rejections - 35 USC § 102 

3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 1 22(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

4. Claims 1, 2, 3, 8-11, and 13 rejected under 35 U.S.C. 102(e) as being anticipated 
by Nakamura et al.(US Patent 6,963,374). 

Consider claim 1, Nakamura et al. teach: 
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An image sensing apparatus("Digital Camera", figures 1-4, column 2, line 56 
through column 4, line 43) comprising: 

an image sensing device("CCD", 303, figure 4) which outputs image data 
obtained by an image sensing element(column 3, lines 50-58); 

a white balance integration device("Black Level Correction/WB", 211a, figure 6a) 
which integrates the image data output from said image sensing device for white 
balance processing(column 5, lines 55-63); 

a display device("EVF", 20, or "LCD", 10, figure 4) which displays an object 
image during imaging on the image sensing element(The display acts as a "live view 
display"(i.e. an object image is displayed during imaging), column 3, lines 16-23.); and 

a control device("main CPU", 21 , figure 4) which causes said white balance 
integration device(211a) to perform integral processing for the image data during read 
of an image signal from the image sensing element(column 5, lines 50-63), and causes 
said display device(21) to display the object image at least after the integral processing 
ends(column 5, line 66 through column 6, line 3). 

Consider claim 2, Nakamura et al. teach: 

An image sensing apparatus("Digital Camera", figures 1-4, column 2, line 56 
through column 4, line 43) comprising: 

an image sensing device("CCD", 303, figure 4) which reads an image signal 
corresponding to an object image from an image sensing element(column 3, lines 50- 
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58), and outputs first image data(First image data corresponds to initial capture data 
which is written as raw data into DRAM(232) of figure 4, column 7, lines 10-18); 

a white balance integration device("Black Level Correction/WB", 21 1a, figure 6a) 
which integrates the first image data(raw data) output from said image sensing device 
for white balance processing(column 7, lines 1 5-1 8); 

a display device("EVF", 20, or "LCD", 10, figure 4) which displays the object 
image during imaging on the image sensing element(The display acts as a "live view 
display"(i.e. an object image is displayed during imaging), column 3, lines 16-23.); 

an image processing devicefimage signal processor", 211, figures 4 and 6a) 
which generates second image data on the basis of the first image data output from 
said image sensing device(Raw data(i.e. first image data) is read out of memory and 
processed by the image signal processor(21 1) to give second image data, column 7, 
lines 24-28.); and 

a control device("main CPU", 21, figure 4) which causes said white balance 
integration device(21 1a) to perform integral processing for the first image data(raw data) 
during read of the image signal from the image sensing element(column 5, lines 50-63), 
causes said image processing device(21 1) to perform image processing of previous 
image data before the first image(column 7, lines 24-28), and causes said display 
device to display the object image during imaging on the image sensing element after 
the integral processing and the image processing end(See figure 7, when the first image 
data processing of ST9 and the previous image data processing of ST10-ST12 are 
completed, the flow chart loops back around to ST2 and ST3 wherein the display device 
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displays the object image. Note: The CPU(21 ) controls all necessary functions of the 
digital camera, column 4, line 1.) 

Consider claim 3, and as applied to claim 2 above, Nakamura et al further teach: 
the first image data includes image data having a signal amount corresponding to 

a color filter of the image sensing element(column 3, lines 50-58, the image signal is 

filtered into red, green, and blue image data), 

and the second image data includes image data capable of confirming the object 

image(column 4, lines 11-19). 

Consider claim 8, and as applied to claim 2 above, Nakamura et al. further teach 
of a temporary storage device("DRAM", 232, and "Memory Card", 8, figure 4) which 
temporarily stores at least two first image data(DRAM(232) reads in raw data(first first 
image data) over channel 1 , and reads out preceding image data(second first image 
data) over channel 2, column 7, lines 15-49. Therefore, both first image data are stored 
in DRAM temporarily.) and one second image data(The memory card(8) stores second 
image data(i.e. processed image data), column 7, lines 34-38). 

Consider claim 9, and as applied to claim 1 above, Nakamura et al. further teach: 
Said control device so controls as to start processing of said image processing 
device(21 1a) at any one of a timing at which a photographing instruction switch is 
released(See figure 7, in step ST1 a shutter release button(i.e. photographing 
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instruction switch) is pressed, and this leads to step ST9 wherein white balance 
processing occurs. See column 6, line 57 through column 7, line 23). 

Consider claim 10, and as applied to claim 1 above, Nakamura et al. further 

teach: 

When display operation of said display device stops("live view display is not 
produced", column 7, lines 18-19), said control device so controls as to start processing 
of said image processing device(211a) at any timing at which a photographing 
instruction switch is released(See figure 7, in step ST1 a shutter release button(i.e. 
photographing instruction switch) is pressed, and this leads to step ST9 wherein white 
balance processing occurs. A live view display is not produced during step ST9. See 
column 6, line 57 through column 7, line 23). 

Consider claim 1 1 , Nakamura et al. teach: 

An image sensing apparatus("Digital Camera", figures 1-4, column 2, line 56 
through column 4, line 43) comprising: 

an image sensing device("CCD", 303, figure 4) which converts light from an 
object into image data(column 3, lines 50-58), and outputs the image data(lnitial capture 
data is written as raw data into DRAM(232) of figure 4, column 7, lines 10-18); 

a display device("EVF", 20, or "LCD", 10, figure 4) which displays the image data 
obtained by said image sensing device(The display acts as a "live view display"(i.e. an 
object image is displayed during imaging), column 3, lines 16-23.); 
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an image processing devicefimage signal processor", 21 1, figures 4 and 6a) 
which performs image processing for the image data obtained by said image sensing 
device(Raw data(i.e. first image data) is read out of memory and processed by the 
image signal processor(21 1 ) to give second image data, column 7, lines 24-28.); 

and a control device("main CPU", 21, figure 4) which causes said image 
processing device(21 1 ) to perform processing for the image data during read of image 
data of a first object from said image sensing device(column 5, lines 50-63), causes 
said display device to display image data of a second object after end of reading the 
image data of the first object(See figure 7, when the first image data processing of ST9 
and the previous image data processing of ST10-ST12 are completed, the flow chart 
loops back around to ST2 and ST3 wherein the display device displays the second 
object.) and so controls as to perform image processing for the image data of the first 
object after end of displaying the image data of the second object(ln step ST10 of figure 
7, image data of a first object is processed after the shutter is released in ST1 to 
capture image data of a second object, column 7, lines 15-49). 

Consider claim 13, and as applied to claim 1 1 above, Nakamura et al. further 
teach that the image processing includes integral processing of image data for white 
balance processing(column 7, lines 15-18). 
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Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 
USPQ 459 (1966), that are applied for establishing a background for determining 
obviousness under 35 U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating 
obviousness or nonobviousness. 



7. Claims 4, 5, 7, 14, and 15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Nakamura et al. in view of Kagle et al.(US Patent 6,967,680). 

Consider claim 4, and as applied to claim 2 above, Nakamura et al. teach of a 
control device that causes integral processing of first image data after causing a display\ 
device to display the object image during imaging on the image sensing element(see 
claim 2 rationale). 

However, Nakamura et al. do not explicitly teach that the apparatus further 
comprises a defect correction device which corrects a defective pixel portion of image 
data when the image sensing element has a defective pixel, and said control device 
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controls said defect correction device so as to correct a defective pixel portion of the 
first image data during the integral processing. 

Kagle et al. is similar to Nakamura et al. in image data is collected from the 
image sensor, preliminary processing is performed to yield first image data, second 
image data is obtained through post processing, and the final image is stored in 
memory(see figure 2, column 3, line 4 through column 4, line 12). Kagle et al. is also 
similar to Nakamura et al. in that white balance p/ocessing is performed during pre- 
processing(column 3, lines 18-23, figure 3). 

However, in addition to the teachings of Nakamura et al., Kagle et al. teach that 
the apparatus further comprises a defect correction device(428, figure 3) which corrects 
a defective pixel portion of image data when the image sensing element has a defective 
pixel(column 3, lines 31-37), and said defect correction device(428) corrects a defective 
pixel portion of the first image data during the integral processing(See column 3, lines 
31-33, defective pixel processing is performed during pre-capture process control(i.e. 
integral processing), step 304, figure 2.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have a defect correction device for correcting defective 
pixels as taught by Kagle et al. in the integral image processing on the first image data 
within the camera device taught by Nakamura et al. for the benefit that the defect 
correction device could determine malfunctioning pixels( column 3, lines 35-37) and 
thereby modify the performance characteristics of the camera in order to correct for 
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defective pixels in advance, thus minimizing any processing delays that are undesirable 
when taking photographs in rapid succession(Kagle et al., column 1, lines 20-47). 

Consider claim 5, and as applied to claim 2 above, Nakamura et al. teach of a 
control device that causes integral processing of first image data after causing a display 
device to display the object image during imaging on the image sensing element(see 
claim 2 rationale). Nakamura et al. also teach of performing white balance processing 
after causing said display device to display the object image during imaging on the 
image sensing element(see claim 2 rationale), and that white balance processing is 
performed on the basis of an integral result of said white balance integration 
device(column 7, lines 15-19). 

However, Nakamura et al. do not explicitly teach that the apparatus further 
comprises a defect correction device which corrects a defective pixel portion of the first 
image data when the image sensing element has a defective pixel, or that a white 
balance coefficient calculation device calculates a white balance coefficient during 
integral processing to be used to perform white balance image processing. 

Kagle et al. is similar to Nakamura et al. in image data is collected from the 
image sensor, preliminary processing is performed to yield first image data, second 
image data is obtained through post processing, and the final image is stored in 
memory(see figure 2, column 3, line 4 through column 4, line 12). Kagle et al. is also 
similar to Nakamura et al. in that white balance processing is performed during pre- 
processing(column 3, lines 18-23, figure 3). 
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However, in addition to the teachings of Nakamura et al., Kagle et al. teach that 
the apparatus further comprises a defect correction device(428, figure 3) which corrects 
a defective pixel portion of image data when the image sensing element has a defective 
pixel(column 3, lines 31-37), and said defect correction device(428) corrects a defective 
pixel portion of the first image data during the integral processing(See column 3, lines 
31-33, defective pixel processing is performed during pre-capture process control(i.e. 
integral processing), step 304, figure 2.). Kagle et al. also teach that a white balance 
coefficient calculation device(404, figure 3) calculates a white balance coefficient(After 
white balance process control is completed, a processing value(i.e. white balance 
coefficient) is returned to the pre-capture process control, column 3, lines 18-27.) during 
integral processing to be used as a basis to perform white balance image 
processing(The processing results of a first frame of image data obtained during pre- 
processing(i.e. integral processing) are used to process the second frame of image data 
as long as the results are within a threshold. See figure 6, column 5, lines 23-44). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have a defect correction device for correcting defective 
pixels, and a white balance coefficient calculation device as taught by Kagle et al. in the 
integral image processing on the first image data within the camera device taught by 
Nakamura et al. for the benefit that the defect correction device could determine 
malfunctioning pixels(column 3, lines 35-37), and the white balance coefficient 
calculation device could produce values which could be applied to later image frames, 
thereby enabling the camera to modify the performance characteristics in order to 



Application/Control Number: 1 0/702,200 Page 1 2 

Art Unit: 2622 

correct for defective pixels in advance, and skip the step of determining a white balance 
coefficient when unnecessary, thus minimizing any processing delays that are 
undesirable when taking photographs in rapid succession(Kagle et al., column 1, lines 
20-47). 

Consider claim 7, and as applied to claim 2 above, Nakamura et al. teach of a 
control device and a display device(see claim 2 rationale). 

However, Nakamura et al. do not explicitly teach that the apparatus further 
comprises a defect correction device which corrects a defective pixel portion of image 
data when the image sensing element has a defective pixel, and said control device 
controls said defect correction device so as to correct a defective pixel portion of the 
first image data before causing said image processing device to start the image 
processing after causing said display device to display the object image. 

Kagle et al. is similar to Nakamura et al. in image data is collected from the 
image sensor, preliminary processing is performed to yield first image data, second 
image data is obtained through post processing, and the final image is stored in 
memory(see figure 2, column 3, line 4 through column 4, line 12). Kagle et al. is also 
similar to Nakamura et al. in that white balance processing is performed during pre- 
processing(column 3, lines 18-23, figure 3). 

However, in addition to the teachings of Nakamura et al., Kagle et al. teach that 
the apparatus further comprises a defect correction device(428, figure 3) which corrects 
a defective pixel portion of image data when the image sensing element has a defective 
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pixel(column 3, lines 31-37), and said defect correction device(428) corrects a defective 
pixel portion of the first image data during the integral processing(See column 3, lines 
31-33, defective pixel processing is performed during pre-capture process control(i.e. 
integral processing, before the start of image processing), step 304, figure 2.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have a defect correction device for correcting defective 
pixels as taught by Kagle et al. in the integral image processing on the first image data 
within the camera device taught by Nakamura et al. for the benefit that the defect 
correction device could determine malfunctioning pixels(column 3, lines 35-37) and 
thereby modify the performance characteristics of the camera in order to correct for 
defective pixels in advance, thus minimizing any processing delays that are undesirable 
when taking photographs in rapid succession(Kagle et al., column 1, lines 20-47). 

Consider claim 14, and as applied to claim 13 above, Nakamura et al. further 
teach that the image sensing device reads out image data of one frame in two 
fields(column 3, lines 50-58). 

However, Nakamura et al. do not explicitly teach that said image processing 
device performs integral processing of the image data before the completion of the read 
in the two fields. 

Kagle et al. is similar to Nakamura et al. in image data is collected from the 
image sensor, preliminary processing is performed to yield first image data, second 
image data is obtained through post processing, and the final image is stored in 
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memory(see figure 2, column 3, line 4 through column 4, line 12). Kagle et al. is also 
similar to Nakamura et al. in that white balance processing is performed during pre- 
processing(column 3, lines 18-23, figure 3). 

In addition to the teachings of Nakamura et al., Kagle et al. teach that said image 
processing device performs integral processing of the image data before completion of 
read in the two fields(Kagle et al. teach that a sub-set of a frame of image data is read 
out and its pre-capture components are verified. The results of this are then applied to 
the rest of the image frame, column 5, lines 23-44.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to process a first field of a frame of image data before 
reading out the second field of image data as taught by Kagle et al. in the apparatus 
taught by Nakamura et al. for the benefit that if the first field of image data is similar to 
the previous frame captured, pre-processing on the second field of image data could be 
skipped by simply applying the values obtained through pre-processing of the previous 
image frame, and thus improving processing speed(Kagle et al.(column 5, lines 23-44). 

Consider claim 1 5, and as applied to claim 1 3 above, Nakamura et al. teach of 
white balance processing(see claim 13 rationale). 

However, Nakamura et al. do not explicitly teach that the image data of the 
second object is processed with the same white balance as a white balance of the 
image data of the first object. 
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Kagle et al. is similar to Nakamura et al. in that image data is collected from the 
image sensor, preliminary processing is performed to yield first image data, second 
image data is obtained through post processing, and the final image is stored in 
memory(see figure 2, column 3, line 4 through column 4, line 12). Kagle et al. is also 
similar to Nakamura et al. in that white balance processing is performed during pre- 
processing(column 3, lines 18-23, figure 3). 

In addition to the teachings of Nakamura et al., Kagle et al. teach that the image 
data of the second object is processed with the same white balance as a white balance 
of the image data of the first object(column 5, lines 23-44). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to process data of the second object with the same white 
balance of the image data of the first object as taught by Kagle et al. in the apparatus 
taught by Nakamura et al. for the benefit of improving processing speed by skipping the 
pre-processing stage for the second image data(Kagle et al.(column 5, lines 23-44). 

8. Claim 6 is rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Nakamura et al: in view of Anderson(US Patent 6,137,534). 

Consider claim 6, and as applied to claim 2 above, Nakamura et al. teach that a 
control device is used to control all functions of the camera(see claim 2 rationale) and 
that integral processing is performed on the image data after the display device displays 
the object image during imaging on the image sensing element(see claim 2 rationale). 
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However, Nakamura et al. do not explicitly teach the apparatus further comprises 
a thumbnail image generation device, which generates a thumbnail image on the basis 
of the first image data. 

Like Nakamura et al., Anderson teaches of a camera(figure 3) containing an 
imaging device(1 14) and a display(402). Anderson also similarly teaches of displaying 
live-view data(61 2, figure 6, column 7, lines 31 -35), and of performing image 
processing(622, figure 6, column 8, lines 15-19). 

However, in addition to the teachings of Nakamura et al., Anderson teaches that 
the apparatus further comprises a thumbnail image generation device which generates 
a thumbnail image on the basis of the first image data(See column 7, lines 22-41 . 
Anderson teaches that the raw data used for live view display(i.e. first image data) is 
used to create a thumbnail image, due to the fact that the thumbnail image need not be 
a high resolution image.). 

Therefore, it would have been obvious to a person having ordinary skill in-the art 
at the time of the invention to include a thumbnail generation device, which generates a 
thumbnail image on the basis of first image data as taught by Anderson in the apparatus 
taught by Nakamura et al. for the benefit that multiple images can be displayed on a 
display for review by a user with thumbnail images(column 1, lines 20-23), thumbnail 
images can provide instant review of captured images(column 1 , lines 49-54), and 
thumbnail images need not be of high resolution and thus can be created from already 
obtained live view display data(column 7, lines 22-41). 
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9. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nakamura et al. in view of Hieda et al.(US Patent Application Publication 
2002/0033887). 

Consider claim 12, and as applied to claim 1 1 above, Nakamura et al. teach that 
the image sensor is read out using interlaced read("interline CCD", column 3, lines 54- 
58).' 

However, Nakamura et al. do not explicitly teach of a switching device which 
switches between read of all pixels as read of the image data of the first object, and 
cumulative read or interlaced read as read of the image data of the second object. 

Like Nakamura et al., Hieda et al. teach of a digital camera(figure 1). Hieda et al. 
also similarly teach that the digital camera contains and image sensor("CCD",1) and 
multiple signal processing areas("Camera Signal Processing Unit", 3, and "Recording 
Signal Processing Unit", 4, figure 1). 

However, in addition to the teachings of Nakamura et al., Hieda et al. teach of a 
switching device(5, figure 1) which switches between read of all pixels as read of the 
image data of the first objectfAt this time...", paragraph 0065), and cumulative 
read(paragraph 0071) or interlaced read(paragraph 0069) as read of the image data of 
the second object(paragraph 0068). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to contain a switching device which switches between read 
of all pixels as read of the image data of the first object, and cumulative read or 
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interlaced read as read of the image data of the second object as taught by Hieda et al. 
in the apparatus taught by Nakamura et al. for the benefit that an image signals with 
good vertical resolution could be obtained through cumulative read, thereby improving 
image quality, and image signals with worse vertical resolution, yet better time division 
resolution, could be obtained through interlace scanning, thereby improving processing 
time(Hieda et al., paragraph 0013). 
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